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l/^ ■ Abstract 

We introduce a new family of concatenated codes with an outer low-density 
parity-check (LDPC) code and an inner low-density generator matrix (LDGM) 
code, and prove that these codes can achieve capacity under any memoryless binary- 
input output-symmetric (MBIOS) channel using maximum-likelihood (ML) decod- 
ed | ing with bounded graphical complexity, i.e., the number of edges per information 
bit in their graphical representation is bounded. We also show that these codes can 
achieve capacity for the special case of the binary erasure channel (BEC) under be- 
lief propagation (BP) decoding with bounded decoding complexity per information 
' bit for all erasure probabilities in (0, 1). By deriving and analyzing the average 

■ weight distribution (AWD) and the corresponding asymptotic growth rate of these 
! codes with a rate-1 inner LDGM code, we also show that these codes achieve the 
' Gilbert- Varshamov bound with asymptotically high probability. This result can be 

■ attributed to the presence of the inner rate-1 LDGM code, which is demonstrated 
to help eliminate high weight codewords in the LDPC code while maintaining a 
vanishingly small amount of low weight codewords. 
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X ■ 1 Introduction 



During the last decade, several codes have been found to achieve capacity on the binary 
erasure channel (BEC) under iterative decoding. The first well-known example is the low- 
density parity-check (LDPC) codes, which were introduced by Gallager [1] and proved 
to be capacity-achieving about forty years later [2,3]. Another prominent example is 
the irregular repeat-accumulate (IRA) codes, whose systematic [4] and nonsystematic [5] 
versions have both been proved to be capacity-achieving. One common feature shared 
by these codes is that they can be understood to be codes defined on bipartite graphs 
with variable nodes and check nodes [6], and their iterative decoding complexity is closely 
related to the number of edges in their graphical representations. A fundamental question 
arises: "How simple can the graphs be as a function of their performance?" 

In [7], the authors give an information theoretical lower bound to show that if all 
variable nodes are transmitted, then the graphical complexity, i.e., the number of edges 
per information bit in the graph, should grow indefinitely as the multiplicative gap to 
capacity decreases to on any memoryless binary-input output-symmetric (MBIOS) 



channels. This is true even if maximum-likelihood (ML) decoding is used. On the other 
hand, allowing state nodes in the graph, the authors in [5] show that nonsystematic 
IRA codes can achieve capacity on the BEC with bounded graphical complexity by 
using the density evolution method [8] . However, partially due to the limitation of the 
density evolution method, whether graphs with state nodes can achieve capacity with 
bounded graphical complexity on more general channels other than the BEC still remains 
unknown. It should be noted that graphical complexity does not translate directly to 
decoding complexity for a general MBIOS channel when an iterative decoder is utilized. 
Indeed, it has been conjectured in [9] that for an LDPC code which achieves a fraction 
1 — e of the channel capacity, the number of iterations for achieving vanishing bit error 
probability grows as 1/e while the average right degree grows as ln(l/e), and thus the 
average decoding complexity per information bit scales as l/eln(l/e). It is thus unclear 
whether by reducing the graphical complexity to a constant the conjectured number 
of iterations will be influenced. This is an open problem for general MBIOS channels. 
Fortunately, at least for the BEC, this question is resolved, since edges in the graph need 
only be visited once when iterative decoding is performed. 

In this paper, we introduce a new family of concatenated codes defined on graphs, 
namely the concatenated low-density parity-check and generator matrix (LDPC-GM) 
codes, and prove that these codes can achieve capacity using ML decoding on any MBIOS 
channels with bounded graphical complexity. These codes are constructed by serially con- 
catenating an outer LDPC code and an inner low-density generator matrix (LDGM) code. 
By deriving and analyzing the average weight distribution (AWD) and its corresponding 
asymptotic growth rate of these codes with a rate-1 LDGM inner code, we show that 
the inner rate-1 LDGM code can help eliminate high weight codewords in the LDPC 
code while maintaining a vanishing small amount of low weight codewords. The result- 
ing AWD of these codes thus has an asymptotic growth spectrum, which can be upper 
bounded by that of the random ensemble in the positive region of the curve while the 
number of codewords in negative region vanishes at least polynomially in n. The ML per- 
formance bound given in [10] is then used to prove our main result. Note that, although 
the ML performance does not translate directly to the iterative decoding performance of 
the codes, the value of this result is twofold. First, there are improved iterative decoding 
algorithms that approach closely the ML performance [11,12]. Thus, it is conceivable 
that the ML performance can be achieved with decoding algorithms having complexity 
close to that of iterative decoding. Moreover, this finding gives a necessary condition for 
achieving capacity with suboptimal iterative decoding algorithms without resorting to 
the density evolution method, which becomes an infinite dimensional problem on chan- 
nels other than the BEC. As a supportive fact on the potential of these ensembles under 
iterative decoding, we also show that these codes can achieve capacity on the BEC with 
bounded decoding complexity per information bit for all erasure probabilities in (0, 1). 

The remaining of this paper is organized as follows. We review and prove some basic 
properties of the AWD of Gallager's LDPC ensemble in Section |21 and derive the average 
input-output weight enumerator of the LDGM codes in Section EJ Then, in Section HI 
we introduce a family of the LDPC-GM codes and give their AWD and the associated 
asymptotic growth rate based on the previous two sections. Detailed analysis on these 
LDPC-GM codes is done and the main result is presented in Section |5J Allowing the 
outer LDPC code and inner LDGM code to be more generally irregular, we prove that the 
LDPC-GM codes can achieve capacity on the BEC with bounded decoding complexity 
in Section Finally, we conclude this work in Section [7| 



2 The Average Weight Distribution of Gallager's 
LDPC Ensemble 

Consider Gallager's (n,j,k) LDPC ensemble as introduced in [13] with guaranteed rate 
R Q — 1 — j/k. Let N a (l) be the average number of codewords of weight / in a randomly 
drawn code from the ensemble. The asymptotic growth rate of N (l) is given in [14] (it 
appears as an upper bound in [13]) to be 

w (a)± lim - In N^an) = { inf (in (1 + g) * + (1 " ^ ) - (j - l)H(a) (1) 

where H(a) = —a In a — (1 — a) ln(l — a) is the binary entropy function evaluated with 
natural logarithms. Some useful characterizations of N a (l) and w (a) are summarized 
below. 

Fact 1 There exists a S a £ (0, 1/2), such that 

i- T,i o Jur) = o{n-^). 

2. w (a) < and has exactly one local minimum, but no local maximum for all a £ 

(OA). 

3. w (a) > for all a £ (5 , 1/2], and w (5 ) = 0. 

4- w (a) has exactly one local maximum at a = 1/2, and w (l/2) = R \n2. 



5. When k is even, N (l) = N Q (n — I), for all I £ {0, 1, . . . , n}. 

In Fact ^ item 1 to 4 are proved in [13, Appendix A], and item 5 follows from the 
linearity of the LDPC codes and the fact that the all-1 word is always a codeword when 
k is even. In order to use item 5, and for other mathematical convenience, we will assume 
throughout this paper that k is even. 

We would like to prove two more results, which will help our later analysis involving 
LDPC codes. The first lemma gives a close-form upper bound on w (a), which is tight 
especially when a is around 1/2. 

Lemma 1 w (a) < (1 - R ) ln[l + (1 - 2a) k ] + [H(a) - (1 - R D ) In 2]. 

Proof: Bounding the infimum term of (0) by substituting x = proves the 
lemma. ■ 
The next lemma gives a sufficient condition on k for any desired lower bound of 5 , 
where we denote by H~ l (x) the unique a £ [0, 1/2], such that H(a) = x. 

Lemma 2 Given any 5 h £ (0,iJ _1 ((l - R a ) ln2)) ; if 

In 



k > 



1 _ g W 

(l-il )ln2 



(2) 



ln(l - 2(5,) ' 
then 5 > Si. 

Proof: After some algebraic manipulations, it can be shown that 
In fx _ 

(1 - R ) ln(l + (1 - 25^) + [H(6i) - (1 - R ) In 2] < (3) 



k > 



ln(l - 25 t ) 

Now, the lemma follows from Lemma ^ and Fact ^ 



3 The Average Input-Output Weight Enumerator of 
the LDGM Ensemble 



Consider the regular LDGM ensemble with codeword length n such that each input node 
is connected to c check nodes, and each check node is connected to d input nodes. Let 
Z Wj h be the average number of codewords with input weight w and output weight h in a 
code drawn randomly from the ensemble. We have 



J w,h 



Rin 
w ) 



)P(H = h\W = w), (4) 



where Ri = d/cis the rate of the LDGM codes, and H and W are random variables 
denoting the input and output weight, respectively, of a randomly drawn codeword. 
Now, given the input weight of the codeword is w, the output weight is h if and only if 
exactly h check nodes are connected to an odd number of edges emanated from "1" input 
nodes, and the remaining n — h check nodes are connected to an even number of them. 
Counting the number of ways of connecting cw edges to dn check node sockets such that 
exactly h check nodes have an odd number of connections, we see that the value is equal 
to 

(£j coef(/_(x, d) h f+(x, d) n ~\ O, (5) 

where coei(f(x), x a ) denotes the coefficient of x a in the polynomial f(x), and f_(x,d) = 
|[(1 + x) d — (1 — x) d ] and f+(x, d) = |[(1 + x) d + (1 — x) d ] are defined to simplify notation. 
Since the total number of ways of connecting cw edges to nd sockets is equal to ( n f\ , we 
have 



fnd\ 
\cwj 



P(H = h\W = w) = ^coef(f4x,d) h U(x,d) n -\x cw ) (6) 



Combining and JBJ), we obtain the average input-output weight enumerator of the 
(c, d) regular LDGM ensemble 



fnd/c\ 
V w ) 



Z ^ = ^[ n h )coeW4x,d) h f + (x,dr- h ,x cw )- (7) 

\cwj ^ ' 

4 Concatenation of LDPC and Rate-1 LDGM Codes 

Consider the concatenation of an outer Gallager's (n,j,kx) LDPC code and an inner 
rate-1 (/c2, k-z) regular LDGM code as shown in Fig. ^ For simplicity, we assume that 
k = ki = k 2 throughout this paper. If we ignore the possibility that different LDPC 
codewords can become the same codeword after further encoded by the inner LDGM code 
and just overcount them, then due to the randomness of the LDPC code construction 
(although we do not assume a uniform interleaver between the inner and outer codes), 
the AWD of the overall code N(l) can be bounded by 



N(l) < N^(l) 4 ^ ^ = Yl ^7# coef (/-(^ k YU(^ k r~ l > * k % (8) 
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Figure 1: The factor graph of the LDPC-GM codes 



where the change of the range of summation in the last equality is due to the fact that 
coef(/_(x, k) l f + (x, k) n ~ l , x ks ) = for s < \l/k~\ and s > \n — l/k\. To calculate the 
asymptotic growth rate of N ub (l), we use the following important equation given in [15] 



hm — In coef( f (x) , x ) = mf In 

n— »oo n ' x >n 

coef(/(x),x a ™)^0 " 



/(*) 



X" 



(9) 



where < a < 1, and f(x) is a polynomial with nonnegative coefficients. Also used is 
the well known property of binomial coefficients 



lim — In 

n^oo n 

© and (HOD then give 



n 
an 



H(a), Vae[0,l] 



(10) 



w a 



: lim — In N (an) 

n~*oo fl 
1 



f^kfUixik) 1 -* 



X 



bk 



< lim -\nN ub (an) 

n^oo n 

=H(a) + max w n (b) — kH(b) + inf In 

a<b<l_a x >0 

k — — A: 

(a) 

<H(a) + a max a w (b) + a\n[l - (1 - 2b) k ] + (1 - a) ln[l + (1 - 2b) k ] - In 2 
=w u \a) (11) 



where (a) follows by substituting x = ^zj i n the infimum expression. To investigate 
the true rate R\ of a randomly drawn LDPC-GM code from this ensemble, let iV(0) be 
the random variable denoting the number of LDPC codewords which after encoded by 
the inner LDGM encoder becomes the all-0 word. Then, we have by linearity of the 
LDPC-GM codes and Markov's inequality that 



N(0) 



P{R X <R Q -r) = P(N(0) > 2 nr ) <^-L<0(2 



■)n(w ub (0)-r)\ 



(12) 



which goes to as n goes to infinity for all r > w ub (0). Therefore we can define the 
guaranteed rate of these LDPC-GM codes with asymptotically high probability to be 



R = R -max{w ub (0),0} 



(13) 



5 Analysis of the LDPC-GM Codes 



In this section, we will first characterize w ub (a), and then use the derived results to 
prove that LDPC-GM codes can be capacity-achieving on the MBIOS channels using 
ML decoding with bounded graphical complexity. Although w ub (a) is not symmetric 
about a = 1/2, the following lemma shows that we can focus on analyzing for 
a G [0, 1/2] and bound w ub (a) by w ub (l - a) for a G [1/2, 1]. 

Lemma 3 w ub (a) < w ub (l - a) for all a G [0, 1/2]. 

Proof: It follows from the facts that 

ln[l - (1 - 2b) k ] < < ln[l + (1 - 2b) k ] \/b G [0, 1] (14) 

and a < 1 — a for all a G [0, 1/2]. ■ 
In the next theorem, we prove that given any R' in [0, 1], the positive part of w ub (a) 
can be upper bounded by if (a) — (1 — R Q ) In 2 if k is sufficiently large for all R Q G [0, R 1 ]. 
In this case, we also prove that N ub (l) at least decreases polynomially with n in the 
negative part of when j > 3. 

Theorem 1 For any R' G [0, 1], there exists an integer M < oo such that for all k > M 
and R G [0, R'], there exists a 5' < H~ l ((l — R Q ) In 2) such that the following two things 
are true. 

1. 

if a = 0, 

if ae (0,61 (15) 
-(1-R )\n2 if a G (5', 1/2]. 




2. N ub {l) = 0{n~i +2 ) for all I G (0, S'n] U[n- S'n, n] . 

Proof: See Appendix \M B 
From the above theorem and the definition of the guaranteed rate of the LDPC-GM 
codes in ()13j) . we have the following corollary, which says that the conditions implied by 
the above theorem also guarantee no rate reduction for the LDPC-GM codes. 

Corollary 1 If k > M, where M is as defined in Theorem^for some R' G [0, 1], then 
R = R a for all R Q G [0,R'\. 

Moreover, if we let d m i n and N(l) be the random variables denoting the minimum 
distance and number of codewords of weight I, respectively, of a randomly drawn code 
from the concatenated ensemble, and let 5qv — — -R)hi2) be the normalized 

Gilbert- Varshamov distance, then from Markov's inequality and Theorem Q we have 

P{d min < S GV n) =P 

< 2J N(l) 

Ze(0,<5 G yn) 




<n max N ub (l) + nexplrt max w u (a) + o(n)\ 
ie{o,6'n] ae{S',s GV ) 

=0(n- j+3 ) (16) 



which goes to asymptotically as n goes to infinity when j > 4. Therefore, we have the 
following corollary. 

Corollary 2 If k > M, where M is as defined in Theorem^for some rate R and j > 4, 
then the LDPC-GM codes have a normalized minimum distance greater than or equal to 
the Gilbert- Varshamov bound with asymptotically high probability. 

In Fig. we compare the asymptotic growth rate of the LDPC, the LDPC-GM and 
the random ensemble with R = 0.5 and k = 8. It is evident that the rate-1 LDGM 
inner code really helps eliminate high weight codewords in the outer LDPC code. As a 
trade-off, the growth rate of some low weight codewords increases slightly. However, as 
long as the growth rate of the low weight codewords remains negative, Theorem ^ shows 

that t^ mr tranicVi a« r> crnfiQ i~ri infmitAr 
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Figure 2: Comparison of w (a), w ub (a) and H(a) 
logarithm is to the base 2 in this figure. 



1 - R) with R = 0.5 and k = 8. The 



We are now ready to state our main theorem, which shows that given any MBIOS 
channel, there always exists a finite value M such that these LDPC-GM codes with 
k > M is capacity-achieving. 

Theorem 2 Given any MBIOS channel with capacity C , there exists an integer M < oo 
such that if k > M, then R = R Q for the LDPC-GM ensembles with R Q < C. Moreover, 
for the given channel, the average block error probability Pb of the LDPC-GM ensembles 
with k > M , j > 4 and R < C is vanishingly small when ML decoding is used. 



Proof: See Appendix IB1 



As can be seen in Fig. ^ the graphical complexity A of these LDPC-GM codes can 
be evaluated as follows. 



= n(j + k)+n = (2-R)k + l 

Rn R 1 ' 

Since Theorem |21 says that k need not go to infinity to achieve capacity, we can deduce 
that these LDPC-GM codes with any rate R G (0, 1) can be capacity achieving with 
bounded graphical complexity. 



6 Density Evolution for LDPC-GM Codes on the 
BEC 

Although the aforementioned LDPC-GM ensembles have finite graphical complexity, the 
decoding complexity under ML decoding is still exponential. In this section, we show 
that by allowing the outer LDPC codes to be more generally irregular, the LDPC-GM 
ensemble can be capacity-achieving on the BEC under BP decoding with bounded de- 
coding complexity per information bit. Although this is not a proof that the same might 
be true for the MBIOS channels, it is a good indication of the potential of the LDPC-GM 
codes. 

Consider the concatenation of a (A, p) irregular LDPC code and a (2, 2) regular LDGM 
code, where A and p are the standard variable and check node degree distributions, 
respectively, from the edge perspective as defined in [16]. Note that this LDPC-GM 
ensemble has guaranteed rate 

R=l- I j p{t)dt (18) 

and our task is to successfully decode the non-transmitted LDPC codewords. Let q be 
the channel erasure probability, and let x±, x 2 , x% and x\ be the probabilities of erasure on 
edges from check to variable(LDGM), variable to check(LDPC), check to variable(LDPC) 
and variable to check(LDGM), respectively as shown in Fig. ^ Then, assuming we are 
operating at some fixed point, we have the following density evolution equations. 

Xl =1- (l-g)(l-x 4 ) (19a) 

x 2 =x\X{x 3 ) (19b) 

x 3 =l-p{l-x 2 ) (19c) 

x 4 =Xi\(x 3 ) (19d) 



where X(x) = YliLi ^i x% an d 



A* = -r^ , 20 



which denotes the fraction of variable nodes in the LDPC code with degree i. Equiva- 
lent ly, we have 

~ s f^X(t)dt , s 

Ax = \ V 7 21 

Jo Kt)dt 



Note that equations (fT5j) are also the density evolution equations for the serially concate- 
nated codes with an outer LDPC code and an inner differentiator code. Solving these 
equations for x 3 , we have 



X3 = p\l- 



?)A(x 3 ). 



A(x 3 ) 



(22) 



If (j22j) has no solution in (0, 1], then x 3 must converge to and thus X4 must converge 
to as the number of iterations goes to infinity. Therefore, if we have 



P 1- 



Q 



(1 - ?)A(x 3 ) 



A(x 3 ) < x 3 , Vx 3 e (0, 1] 



(23) 



then the BP decoding is successful. Note that ((221) * s essentially the same as equation (6) 
in [5] except for the following changes: Xq — > 1 — x 3 , p — > 1 — q, A(-) — > p(-), p(-) — > A(-), 
and /?(•) — > A(-). More generally, ()22|) is an instance of the symmetry introduced in [17]. 
So, in the following, we will use the results proved in [5] to show two particular degree 
distribution pairs are capacity-achieving under BP decoding. 



Theorem 3 (Check-regular ensemble) Let 

1 - (1 - x)i& 



X(x) 



p(x) 



-X 



1- (1-q) (l-kx + (k-l) 
fc-i 



1 - (1 -x)& ) 



(24) 



(25) 



Then for k = 3 and q £ [-^, 1), X(x) has only non-negative coefficients. Moreover, for 
any e 6 (0, 1), let M(e) be the smallest positive integer such that 1 



E 



t=Af(e)+l 



Ai ^ e(l - q) 
i qk 



(26) 



and let A e (x) 6e t/ie truncated degree distribution of \{x) by treating all variable nodes with 
degree greater than M(e) as pilot bits. Then the degree distribution pair (A e ,p) achieves 
a fraction 1 — e of the channel capacity with vanishing bit error probability under BP 
decoding. 

Proof: See Appendix IC.ll ■ 
The decoding complexity per information bit of this check-regular ensemble can be cal- 
culated as follows 



A < 



knq + 2n + n 



qk + 3 



(l-g)(l-e)n (l-q)(l-e) 

which approaches the bounded value as e goes to 0. 

j_ 

qk 



(27) 



1 M(e) exists for all e G (0, 1) since YliLi T 1 = Jq 1 Ht)dt = which means 5Zi^M(e)+i T 1 can ^ c 



made arbitrarily close to by increasing M(e). 



Theorem 4 (Variable-regular ensemble) Let 



X(x) =x 2 (2* 
2(l-g)(l-x) 2 sin (|arcsin { J- 27(1 ~g 3 1 ~ a:) § 



p(x) =1 + 



9 3 



(29) 



T/ien /or g G [0.05,1], p(x) /ias on/?/ non-negative coefficients. Moreover, for any e G 
(0, 1), let M{e) be the smallest positive integer such that?' 



e(l -g) , . 

Pi < — o — ( 30 ) 



i=M(e)+l 

and let 



Pe{x) = 



M(e) \ Af(e) 



&e i/ie truncated degree distribution of p(x). Then the degree distribution pair (A, p e ) 
achieves a fraction 1 — e of the channel capacity with vanishing bit error probability under 
BP decoding. 

Proof: See Appendix 1(121 ■ 
The decoding complexity per information bit of this variable-regular ensemble can be 
calculated as follows 

3n + 2n + n 6 
< (l-q)(l-e)n (l-g)(l-e) 1 ' 

which approaches the bounded value as e goes to 0. 

One drawback of the these capacity-achieving degree distribution pairs is that they 
are not guaranteed to be valid, i.e., with only nonnegative coefficients, for all q G (0, 1). 
However, since they are valid for q close 1 (which is not true for the capacity-achieving 
IRA codes in [5]), this problem can be solved by considering punctured LDPC-GM codes. 
In [18], it is shown that random puncturing results in no performance loss on the gap 
to capacity for codes on the BEC. Hence, it follows that puncturing can be used to 
increase the rate of the LDPC-GM codes without affecting its capacity-achievability, a 
fact that was also observed by Pfister and Sason [17]. Furthermore, since a punctured 
LDPC-GM ensemble can also be viewed as another unpunctured LDPC-GM ensemble 
with inner irregular LDGM codes (which is no longer rate-1 in general), we have the 
following theorem. 

Theorem 5 Let (A,p) be a degree distribution pair implied by Theorem^ or Theorem^ 
for some given e and q' . Consider the LDPC-GM ensemble, whose outer LDPC code has 

2 M(e) exists for all e € (0, 1) since Pi = L which means X^M(e)+i Pi can ^ e ma de arbitrarily 

close to by increasing M(e). 



degree distribution pair (A, p) from the edge perspective, and inner LDGM code has degree 
distribution pair (F, G) from the node perspective 3 . Then for any given p G [0, q'}, if 

F(x) =[x(l -p) +p} 2 (33) 
G(x) =x 2 (34) 

then this LDPC-GM ensemble achieves a fraction ofl — e of the channel capacity on the 
BEG with erasure probability q = ^2 under BP decoding. 

Proof: See Appendix IC.3I ■ 
This theorem says that, given any capacity-achieving degree distribution pair (A,p) for 
some erasure probability q', we can generate capacity-achieving LDPC-GM ensembles 
for all erasure probabilities q G [0, q'] by adjusting p. Since q' can be arbitrarily close to 
1, and the maximum degrees of F and G are bounded for all p, this construction can 
be done to produce capacity-achieving LDPC-GM ensembles for all rate in (0, 1) on the 
BEC with bounded decoding complexity. 



7 Conclusion 

In this paper, the LDPC-GM codes, i.e., the concatenated codes with an outer LDPC 
code and an inner LDGM code, are introduced. In the case that the outer code is 
Gallager's (n, j, k) LDPC code and the inner code is a rate-1 (k, k) regular LDGM code, 
we prove that for any desired range of rates R Q , there always exists an integer M < oo 
such that if k > M then the inner LDGM encoder results in no rate reduction for the 
outer LDPC code. Moreover, the LDGM encoder helps eliminate high weight codewords 
while maintaining a vanishingly small amount of low weight codewords in the LDPC code. 
The resulting asymptotic growth spectrum of the LDPC-GM codes has a positive part, 
which can be upper bounded by the asymptotic growth spectrum of the random ensemble, 
and a negative part, where the number of codewords vanishes at least polynomially in 
n when j > 4. Note that, the condition j > 4 is automatically satisfied when k is big 
enough. It then follows easily that these codes achieve the Gilbert-Varshamov bound 
with asymptotically high probability. Furthermore, after applying the ML performance 
bound given in [10] to these LDPC-GM codes, we prove that they can achieve capacity 
on any MBIOS channels using ML decoding. Since all these results are implied by the 
only condition that k is greater than some finite number, which shows that the number 
of edges per information bit in the graph need not go to infinity to achieve capacity, 
we have proved that these LDPC-GM codes are capacity-achieving codes with bounded 
graphical complexity on any MBIOS channels. 

On the other hand, if the outer LDPC code is allowed to be irregular, then invoking 
the density evolution method, we use the results in [5] to show two particular ensembles of 
the LDPC-GM codes can achieve capacity on the BEC under BP decoding with bounded 
decoding complexity. Moreover, extensions valid for all erasure probabilities of the BEC 
using inner irregular LDGM codes are also presented. These favorable results could 
suggest high potential of the LDPC-GM codes to achieve capacity on the MBIOS channels 
with bounded decoding complexity per iteration. However, since the scaling on the 
required number of iterations for successful iterative decoding for the LDPC-GM codes 

3 That is, F(x) = ^2°Z F%x i and G(x) = X^Li GiX 1 , where Fi and G; denote the fraction of input 
and check nodes that have i neighboring check and input nodes, respectively, in the LDGM code. 



compared to other existing codes remains unknown, whether this bounded graphical 
complexity property implies bounded decoding complexity using iterative decoding is 
still an open problem on MBIOS channels. 



A Proof of Theorem U 

1. Define 

f(b)±w (b) + a\n — 



[l-2b) k . 1 + (1 

> + (1 - a) In v 



2b) 1 



(35) 



We will bound f(b) in two cases. By LemmaEl for any Si G (0, H 1 ((1 — R') In 2)) C 
(0,H-\(l-R o ) ln2)),if 



In 



k > M x 



(l-i?')ln2 



hi 



ln(l - 25, 



> 



H(6i) 



(l-i? )ln2 



ln(l - 25, 



(36) 



then w {5i) < for all R Q G [0,R'}. Therefore, for k > M x and b G [a/k,5{\ U [1 - 
5i, 1 — a/k] (we assume without loss of generality that a/k < 5i. otherwise, we just 
skip this step), we have 



f{b) < w (b) - H(a) < max{w (a/k), w Q (5i)} - H(a) 



(37) 



where the first inequality follows from the fact that relative entropy is always non- 
negative, and the second inequality follows from Fact ^ On the other hand, when 
b G (5i, 1 — 5i), we have from Lemma ^ that 

/(&) <(1 - Ro) hi[l + (1 - 2b) k ] + H{b) - (1 - R ) ln2+ 



+ a In 



2b) 1 



(1 - a) In 



1 + (1 - 2bf 



2 y 2 

< - (1 - Ro) In 2 - In 2 + {H(b) + (2 - R - a) ln[l + (1 - 2b) k ]} 

< - (1 - i? G )ln2 - In 2 + {H(b) + 21n[l + (1 - 2b) k ]} 
where the last two inequalities follow from (|14|) . Since 

8 2 H(b) 1 



8b 2 



b)b 



< -4, 



(3? 



(39) 



and 



d 2 21n[l + (1 - 26) fc ] 8A;[A; - 1 - (1 - 2b) k }(l - 2b) 



k-2 



db 2 



[1 + (i _ 2b) k ] 2 
<8k(k- 1)(1 -26) fe " 2 



<8A;(fc- 1)(1 -25^ fe ~ 2 



(40) 



which can be made arbitrarily close to for a large enough k, there exists a M 2 
such that k > M2 implies that the maximum of H (b) + 2 ln[l + (1 — 26) fc ] is attained 
at b = 1/2, and thus 



/(&)< -(1-^)^2, V6 6(^,1-5, 



(41) 



Therefore, for all k > M = max {Mi, M 2 }, we have 
w u \a) = H(a) + max /(&) < max{#(a) - (1 - # ) In 2, w (a/d), w (8i)} (42) 



Since max{w (a/(i), w (5/)} < for all a > 0, there must exist a 8' < H 1 ((1 — 
R ) In 2) such that this part of the theorem is true. 

2. For all I G (0, 8'n] U [n — 8'n, n] and > M, we have 



L^/fcJ AT ( n \ ^( LDPG ) 



N o{s)Z\ L , 

Nf{l)= 



s=\l/k~\ 

< E E E — 

s=|~Z/fc] s=n— 5jti s=Sin \sJ 

(b) 

<0(n~ 3+2 ) + nexp{n[#(//n) + max /(&)] + o(n)} 

<5i<&<l-<5i 

(c) 

<0(n" J+2 ) + nexp{n[#(//n) — (1 — i2) In 2] + o(n)} 
( io(n- j+2 ) (43) 



where o(n) denotes some value that converges to as n goes to infinity. In 

(a) follows from the fact that Z < f J ® P( ~* ) / (™) < 1 since it is a probability as shown 
in (J3J), (b) follows from Fact^ (c) follows from (|41|). and (d) follows from the fact 
that 8' < H~\(l-R) ln2). 



B Proof of Theorem [21 

Let M be as defined in Theorem ^ for R' = C. Then by Corollary Q we have R = R Q 
for all k > M and R Q < C. Let U C {l,2,...,n}, and U c be its complementary set. 
The following upper bound on the average block error probability under ML decoding is 
given in [10] 

Pb < ^{N(})D 1 } + 2~ nEriR+ ^ (44) 
leu 

where 



a N(l) 2 n , , 

a = max — j-r- (45) 

/G[/C 2*R - 1 (") 

£>•(•) is the random coding exponent, and 

D = E VP^OM^l) < 1 (46) 

y 

where p(y\0) and p(y\l) are the conditional probability density functions of the output of 
the MBIOS channel given the input. If we apply this bound to the LDPC-GM ensemble 
with k > M and R Q < C, and let 

U^ilde(0,8'}U[l-8',l}\, (47) 



where 5' is as defined in Theorem ^ then we have from Theorem^ that 

J2{W)D 1 } < 5^iV^(0 < nO(n~ j+2 ) = 0(n^ +3 ) (48) 
i&u i&u 

But we have from the same theorem and Lemma El that 

lim — = max w(a) - \H(a) - (1 - R) In 21 

n-*oo n ae(S',l-S') 

< max w ub (a) - \H(a) - (1 - R) In 2] 

oe(<5',l/2] 

<0 (49) 

Hence we have 

P B < 0(n- j+3 ) + 2- nE ^ (50) 

which goes to as n goes to infinity for all R < C and j > 4. Thus, the theorem is 
proved. 

C Proofs of Section El 

First, we need a lemma. 

Lemma 4 If the degree distribution pair (A, p) satisfies p(0) = 0, p(l) = 1, and satis- 
fies (E2I) for all x 3 E [0, 1], then R=l-q. 

Proof: [5, Lemma 1] shows that under the assumed conditions, we have 

/o p{t)dt _ 



f*\(t)dt 



(51) 



C.l Proof of Theorem [31 

The facts that (A, p) satisfies ()22|) for all x G [0, 1] and that X(x) has only non-negative 
coefficients for k = 3 and q E [y|, 1) are proved in [5, Theorem 1]. By the definition of 
A e , we have effectively 

M(e) 

K(x) = J2^ l ~ 1 (52) 

i=l 

in the density evolution equations. Hence, it follows that X e (x) < A(x), and the corre- 
sponding X e (x) < A(x) for all x G (0, 1]. Therefore, is satisfied, which implies that 
the BP decoding is successful. To find the rate of this ensemble of codes, let 



oo 
M(e)+1 



(53) 



be the fraction of pilot nodes. Then, we have 

(1 - 6) L l X(t)dt - C p(t)dt 



R 



--1-5 



Jo Kt)dt 
So P^) dt 



Jo' Kt)dt 

=l-q-S (54) 

where the last equality follows from the facts that p(0) = 0, p(l) = 1, and Lemma |U 
But, from flUJ) 



6= , ' l = q E i = 1 k E V«' < e(l - (f) (55) 

M(e)+1 io ^(*)^ M(e)+1 Jo P^)^ Af(e)+1 

Therefore, it follows that i? > (1 — e)(l — g), and the theorem is proved. 
C.2 Proof of Theorem H 

The facts that (A, p) satisfies (|22|) for all x £ [0, 1] and that p(x) has only non-negative 
coefficients for q £ [0.05,1] are proved in [5, Theorem 2]. Since p e (a;) > p(x) for all 
x £ (0, 1], (|23|) is satisfied and the BP decoding is successful. As for the rate of this 
ensemble of codes, we have 



R=l 



>1 



Jo Pe(t)dt 

Jo Ht)dt 

/oA(t)* 

V°° £i _i_ 1 _ V M ( e ) n 

Jo Kt)dt 

Jo pm + e: =M(e)+l P % 

Jl ^)dt 



(«) 



1-9-3 E # 

i=M(e)+l 

>(l-6)(l-g) (56) 

where (a) follows from the facts that p(0) = 0, p(l) = 1, and Lemma 0] Hence, the 
theorem is proved. 

C.3 Proof of Theorem E] 

Let / be the degree distribution corresponding to F from the edge perspective. We have 

m = ^ = x(l-p)+p (57) 



and the following set of density evolution equations 



Xl =1 - (1 - q')(l - z 4 ) (58a) 

x 2 =F( Xl )X(x 3 ) (58b) 

x 3 =l-p{l-x 2 ) (58c) 

x 4 =f( Xl )\(x 3 ) (58d) 

After some algebraic manipulations, the fixed point equation can be shown to be 

q'(l —p)+p 



x 3 = 1 - p 1 



(l-q')(l-p)X(x 3 ) 



A(x 3 ) (59) 



which is the same as (122)1 if we let g be as defined in this theorem. Hence, from Theorem|3] 
and Theorem the decoding is successful under BP decoding on the BEC with erasure 
probability q. Moreover, the rate of this ensemble is given by 

_ , ,. , T -j -j — ^ , , {number of input nodes in the LDGM code} 

R ={rate of the outer LDPC code} x 



={rate of the outer LDPC code} x 
>(l-e)(l-q' 



{number of check nodes in the LDGM code} 
G'(l) 



F'(l) 

1 



1 — p 

=(l-e)(l-q), (60) 
which then proves this theorem. 
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